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Abstract 


The rising cost of college and demands for accountability have increased interest in providing 
students with information about the earnings of college graduates by school and major. However, 
no consensus exists over how to display that information in a way that is most beneficial for 
students. Some researchers advocate displaying median earnings only, while others advocate 
showing more detail on the variation in earnings. We argue that an explicit theory of student 
choice is missing from discussions about the provision of earnings data. We use a survey 
experiment to assess two models of student choice: one in which students use median earnings, 
and one in which students use median earnings and earnings variation. We demonstrate that 
showing respondents the median and variation leads to large and significantly different 
expectations in earnings and different choices in majors, compared to respondents who see the 
median only. The results question the use of medians only as a tool to improve student decision 
making. In contrast, displaying medians and variation provides influential infonnation that allows 
students to make educational choices that incorporate risk. 



1 Introduction 


In the United States, it is widely believed that a college education is necessary to remain 
competitive in the modern economy. College graduates on average have higher lifetime earnings 
(Angrist and Chen 2011; Carnevale, Rose, and Cheah 2011; Oreopoulos and Petronijevic 2013) 
and lower unemployment rates compared to those with no college degree (Abel, Deitz, and Su 
2014; Grusky, Red Bird, Rodriguez, and Wimer 2013). But with federal government estimates of 
student loan debt surpassing one trillion dollars (Chopra 2013), combined with diminishing 
employment prospects of college graduates (Abel et al. 2014; Stone, Van Horn, and Zukin 2012), 
there has been widespread interest in providing more information to consumers about the labor 
market outcomes from college (Jacobson and LaLonde 2013; Johnson, Majia, Ezekiel, and 
Zeiger 2013; Long 2010; Moore, Chapman, Huber, and Shors 2013; Mullin and Lebesch 2010; 
Owen and Sawhill 2013). The goal of this infonnation is to improve the decision making 
regarding the colleges students attend and the majors they choose. 

Despite the potential importance of labor market outcomes for student decision making, 
little is known about how this information influences those who use it. While all agree that the 
data shared with students must be relevant to the decisions they need to make, research has yet to 
identify the type of information that is most relevant for students making educational decisions. 3 

3 

The content of the information displayed differs from the form in which it is presented, its design, and how it is 
displayed, which also has important implications for how students process and act upon the information. The 
focus of this paper is exclusively on what to display to students, leaving the question of how to best display it to 



Figuring out the optimal information to present to consumers is a challenge, because there are 
many quantities of interest, such as the mean, median, and inter-quartile range, that could be 
presented at varying levels of specificity for different sub-groups of students at different times 
after they have left college (Moore et al. 2013; Voight, Long, Huelsman, and Engle 2014). In this 
study, we examine the specific issue of the type of statistic to present on the labor market 
earnings of college majors: median earnings alone or the median combined with earnings 
variation. 4 Median earnings alone, or other measures of central tendency such as average 
earnings, conveys information in a simple manner on the most typical earnings outcome for a 
given major, while the average combined with variation conveys infonnation on the full range of 
possible earnings, providing students with relevant infonnation on the risk associated with a 
given major. The issue of whether to present information on the median or the variance is under 
debate among analysts involved in making these data more available to consumers (Hershbein 
and Hollenbeck 2014; Jacobson and LaLonde 2013). 

To assess how the presentation of different types of earnings infonnation influences 
student educational choices, we conduct a survey experiment with a national convenience sample 
of U.S. adults. The experiment allows us to manipulate the infonnation respondents see and, 
through random assignment, eliminate the confounding influence of unobserved factors 
associated with treatment assignment and outcomes. We compare the effect of presenting median 
earnings with the effect of presenting the median plus the variation in earnings on educational 
choices. We find large and statistically significant differences across experimental groups. 
Compared with respondents who were shown only the median earnings, respondents who 


future research. 
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We use median throughout the paper for consistency and because the median is less sensetive than the average 
to the influence of individuals with extremely high and low earnings. 
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observed the variation in earnings were significantly more likely both to select the major with 
higher variation in earnings and to estimate a higher probability of earning above the median. 

The paper proceeds as follows. In Section 1, we provide background on the policy 
context, review the related literature on earnings and major choice, and discuss our empirical 
expectations. In Section 2, we describe the experimental design of our study and use 
administrative data to compare different visual displays of earnings outcomes. Section 3 presents 
the results of the experiment, while the Section 4 discusses the findings. Section 5 concludes the 
paper. 

1.1 Policy Context: The Push for Labor Market Outcomes Data 

Despite rising costs, a college education remains a good investment (Abel, Deitz, and Su 
2014). Yet, because of the rising costs of college, it has become increasingly important for 
students to make the right choices about where to attend college and what to study, leading to a 
national push to provide students with information, especially information on the outcomes of 
college graduates, to help them make better-informed educational choices. The push to offer 
information about college outcomes is perhaps best illustrated with the Obama Administration’s 
College Scorecard. The College Scorecard aims to improve the transparency and accountability 
of higher education by providing financial information about the costs and outcomes associated 
with colleges so students “can choose a school that is affordable, best-suited to meet their needs, 
and consistent with their educational and career goals” (Obama Administration 2014). The 
College Scorecard seeks to provide employment information on graduates by college as part of 
its broader effort to provide information to guide consumer choice in higher education through 
the College Affordability and Transparency Center (Obama Administration 2014). 
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Likewise, several state-level efforts promote the availability of college outcomes data to 
consumers. State reporting systems make infonnation from their institutions on student outcomes 
publically available, and many of these systems include labor market outcomes. For example, 
California’s community college system developed a reporting system that makes employment 
outcomes data available to the public. 5 Likewise, Texas provides a vast amount of data on higher 
education outcomes on its gainful employment website. 6 Other states, including Florida, 
Minnesota, Virginia, and Washington, have data on median earnings available at the program 
level (Moore et al. 2013). While not yet available across all states, labor market outcomes data 
are becoming increasingly available nationwide, as many states receive federal funds to support 
the linkage of their Unemployment Insurance wage records with other state systems, including 
higher education (Jenkins and Harmon 2010; U.S. Department of Labor 2014). 

Private and nonprofit organizations are also promoting the availability and use of labor 
market outcomes data in decision making. Two prominent efforts are the websites from 
CollegeMeasures.org and College Reality Check. CollegeMeasures.org provides information on 
college outcomes nationally, and has worked with several states, including Arkansas, Colorado, 
Florida, Tennessee, Texas, and Virginia, to assist them in making earnings data on their graduates 
publically available. 7 College Reality Check provides information on college outcomes 
nationwide to assist in decision making; this website uses earnings data from a company named 
PayScale to provide earnings information and prompt students to consider earnings by posing the 


5 See http://datamart.cccco.edu/Outcomes/System_Wage_Tracker.aspx. Last accessed September 4, 2014. 

6 See http://www.txhighereddata.org/reports/performance/ctcasalf/gainful.cfm. Last accessed September 4, 2014. 

7 See www.collegemeasures.org. Last accessed September 4, 2014. 
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question: “Will I make enough to repay my debt?,” and then showing information on earnings at 


starting and mid-career points. 8 

While these data infonn multiple stakeholders, calls for increased availability of data have 
focused heavily on their use in informing consumer decision making. 9 Labor market outcomes 
data are primarily intended to inform the choice of which college to attend and which college 
major to select in order to eventually secure employment consistent with a student’s goals and 
expectations (Herndon 2012; Hershbein and Hollenbeck 2014; National College Access Network 
2013; Owen and Sawhill 2013; Voight et al. 2014). Better labor market infonnation may also 
improve chances of enrollment (Aspen Institute 2013; Morgan, Leenman, Todd, and Weeden 

2013) , as well as persistence in college (Aspen Institute 2013; Stuart, Rios-Aguilar, & Deil-Amen 

2014) . 


Current efforts to provide labor market outcomes data present different statistics to 
summarize earnings information. Two positions about the display of earnings data are prominent 
in writings on student outcomes and labor market infonnation. First, one widespread approach is 
to display the median earnings by major. For example, salary infonnation that allows for the 
comparison of earnings across majors is available at websites like PayScale’s College Salary 
Report and in reports such as Carnevale and Cheah (20 13). 10 When researchers present 
justifications for showing only the median, they argue that the median is a simple, easily 


g 

See www.collegerealitycheck.com. Last accessed September 4, 2014. 

9 

Data on the labor market outcomes are important to several stakeholder groups with different potential uses for the 
data. These groups include consumers (i.e., students and parents) who can use the data to guide their decision 
making, policymakers and the public who can use the data to assess the outcomes on public investments in education 
and adjust policy accordingly, and institutions that can use the data for performance improvement (Aspen Institute 
2013; Voight, Long, Huelsman, and Engle 2014). 

10 See http://www.payscale.com/college-salary-report-2014. Last accessed September 4, 2014. 
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interpretable measure of the expected earnings in a major; additional statistics may overwhelm 
students by providing too much information (Baum, Kurose, and Ma 2013; Jacobson and 
LaLonde 2013; Schneider 2013). 

Other analysts argue that variation in earnings in addition to the median provides useful 
infonnation for students and should be provided (Moore et al. 2013; Mullin 2013). Advocates of 
this approach argue that students and parents can be misled about the expected earnings without 
knowing about the distribution of earnings (Hershbein and Hollenbeck 2014). Variation provides 
students with greater information about the nature of potential earnings across fields — some 
fields have a wide distribution of earnings and others have a more tight distribution of earnings. 
This information might enhance students’ ability to predict their own potential future earnings 
and thus make better decisions about their major and/or student loan debt (Avery and Turner 
2012 ). 

Displaying the variation in earnings may encourage students to reflect on the personal 
characteristics that may influence their future earnings. Do they think that they have the ability to 
be a high performer in their selected field of study, or might they end up on the lower end of the 
earnings distribution? Will they be able to obtain employment in their field of study that has high 
earnings because of their knowledge and social connections in the field? Alternatively, some 
analysts have expressed concerns that the variation in earnings may misinform students as they 
may focus too much on the high end of the distribution and be overconfident in their eventual 
earnings (Hershbein and Hollenbeck 2014). 

Ongoing efforts to make labor market outcomes data available represent attempts to 
provide information where students have traditionally had little to no information (Herndon 
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2012). Data on labor market outcomes have simply been less available for higher education than 
other outcomes such as completion of degree (Voight et al. 2014). As such, these data are being 
introduced into a low-information environment where they could potentially have significant 
impact on the students’ knowledge of labor market outcomes. Although the magnitude of the 
potential effect is large, little is known about the impact of this infonnation on students. 

While efforts to make labor market outcomes data publically available to consumers 
continue to expand, knowledge of how the display of labor market information affects student 
decision making is still under development and in need of theoretical motivation. Rigorous 
testing based on theory-driven expectations will help clarify how the presentation of different 
statistics summarizing labor market infonnation actually informs student choice. 

1.2 Prior Research on Earnings Information and Decision Making 

Our work builds on a considerable theoretical and empirical literature, primarily in 
economics, detailing the relationship between earnings and educational choices. Much of this 
work relates educational choice to expected future income of the major. Willis and Rosen (1979), 
for example, show in a widely cited study that U.S. veterans’ predicted income significantly 
influences their decision to attend college. Warrick, Daniels, and Scott (2010) examine the 
association between expected income and preferences over public- and private-sector accounting 
careers. Other scholars have noted this same pattern; future expected earnings influence the 
decision to attend college, and the decision on what to choose as a major (Altonji, Blom, and 
Meghir 2012; Arcidiacono, Hotz, and Kang 2012). Research has also shown that earnings 
infonnation influences both students’ choices and subjective expectations about their earnings 
potential. In their survey of New York University undergraduate students, Wiswall and Zafar 
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(2013) find that students use earnings infonnation to update their own expected earnings, and 
these revised earnings expectations influence major choice. 

Beyond expected earnings associated with majors, infonnation on the variation, or risk, in 
earnings associated with majors is another dimension to inform decision making. Education is an 
investment in human capital (Becker 1994), and knowing the risk associated with an investment 
is as important as its expected return. Making investment decisions without incorporating the 
riskiness of the investment is at odds with most financial theories of investment. Investment 
decisions in financial economics regularly incorporate expected earnings and risk. 

Investment theory uses assumptions that investors optimize asset portfolios with mean 
and variance of their portfolios. One of the most influential models is the Markowitz 
mean/variance optimization analysis (Markowitz 1952). Under this framework, the investor seeks 
to maximize the expected return and standard deviation of return of the function d = e - v/t, 
where d is the investor’s preference for the portfolio, e is the expected return, v is the variance of 
return, and t is the investor’s risk tolerance. This simple model illustrates the role of variance and 
risk aversion in investment decisions. As v increases or t decreases, the preference for the 
portfolio decreases. It provides an intuitive representation of investment decisions. Educational 
choices are also investment decisions made under uncertainty. Yet, students often receive 
information that shows the expected return of educational choices. In other words, an assumption 
is made that students use an investment model of the following form d = e, instead of d = e - v/t. 

Scholars have begun to explore how risk influences educational choices. In their review 
of educational decision making, Altonji, Bloom, and Meghir (2012) note that the variance of 
earnings — as a measure of risk — should be an important consideration for students when 
considering earnings infonnation. Zocco (2009) studies risk in the context of student course 
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selection. Using a sample of Danish students, Nielsen and Vissing- Jorgensen (2006) provide 
empirical evidence that students prefer educational choices for fields with high median earnings 
and low risk. Likewise, Attanasio and Kaumann (2012) find evidence that perceived risk of 
earnings, as measured by the variance of future earnings, influences the decision to attend college 
among poor Mexican youth. Moreover, they find that perceived risk is particularly important for 
the parents of students. 

More recently, Wiswall and Zafar (2013), based on subjective assessments of the risk 
associated with given majors, find that students exhibit risk-averse behavior in the choice of 
majors. They assess risk by showing students the range of earnings by major. Infonnation on risk 
may be particularly relevant to low-socioeconomic-status students, who are more likely to choose 
majors with less risky post-graduate earnings outcomes and often know less about the labor 
market (Betts 1996; Saks and Shore 2005). 11 Given this research showing the importance of risk 
in educational decision making, developers of educational scorecards must consider the effects of 
including risk when presenting information about post-graduate earnings. 

In understanding how students make sense of earnings information, it is important to 
recognize that numerous factors can influence their expectations about future earnings, including 
a mix of private infonnation (e.g., ability in a major) and public information (e.g., earnings data). 
Private infonnation in the fonn of academic ability has an important relationship to major choice, 
occupational choice, and earnings expectations. High-ability individuals are attracted to the 
subject matter associated with higher-earning majors and high-earning jobs after graduation 
(Arcidiacono 2004). Greater performance, or academic achievement, reflects human capital 
acquisition and thus should be correlated with earnings; that is, we would expect students with 

1 1 Rouse (2004), however, finds no difference by socioeconomic status in knowledge of the labor market outcomes 
associated with different levels of education. 
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high academic achievement to have higher earnings than those with lower academic achievement 
(Becker 1994). Whether this knowledge is salient to students is not well known, though Wiswall 
and Zafar (2013) find a high correlation between students’ self-assessments about ability and 
earnings expectations. When assessing information on the risk of a given major, through the 
variation in earnings, students’ understanding of the relationship between academic achievement 
and earnings is of particular relevance, as we discuss in the following section. 

1.3 Empirical Expectations 

In this study, we assume that individuals update their expectations about future earnings 
when provided with accurate information about the earnings of graduates, as found by Wiswall 
and Zafar (2013). We then investigate how different types of infonnation affect earnings 
expectations and decision making with regard to the choice of major. Using the findings from the 
above discussion of theoretical and empirical work, we fonn two empirical expectations for our 
experiment: 

1 . Earnings information that includes variation leads to different educational choices 
compared to earnings information that includes median earnings alone. 

2. Earnings information that includes earnings variation leads to different expectations about 
earnings compared to earnings infonnation that includes the median only. 

The first expectation denotes that we expect to find differences in the choice of major between 
those who see the median and variation and those who see the median only. The differences in 
choice arise because of differences in earnings expectations between the two groups. The second 
expectation denotes that we expect to find differences in the earnings expectations between those 


10 



who see median and variation and those who see the median only. The difference arises due to 
varying risk preferences, decision making biases, and private infonnation. 

We now present a third empirical expectation: 

3. High-academic performers are more likely to choose the high variation major compared to 
low-academic performers 

The third expectation is intended to assess how private infonnation about a student’s 
academic ability influences the use and interpretation of labor market outcomes data. In 
particular, we expect that high academic ability students will be more likely to form earnings 
expectations toward the high end of the earnings distribution. In contrast, low academic ability 
students will be more likely to fonn earnings expectations near the bottom end of the earnings 
distribution. The high academic ability student is then more likely to choose the high variation 
major, since that choice offers greater potential earnings than the low variation major. 

Meanwhile, the low academic ability student is more likely to choose the low variation major, 
since that choice offers less potential risk of achieving a low-earnings outcome. 

2 Data and Experimental Design 

We conducted a survey experiment in which individuals are randomly chosen to view 
different information about labor market outcomes. The survey begins by asking respondents to 
read a vignette about a fictional college-bound student, Steve. 12 Respondents read about Steve’s 
academic background and interests, and then view earnings outcome information about two 

12 

“ We choose a traditionally male name rather than a traditionally female name in order to focus on the effect of 
labor market information. Using a female name may prime respondents to consider perceived and real gender 
biases in risk aversion, labor market outcomes, and academic performance. These are interesting and important 
considerations for future work. 
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college majors that we chose to highlight differences in the variation in earnings. After viewing 
the earnings outcomes, respondents are asked to make estimates about Steve’s post-graduate 
earnings and recommend which major Steve should choose. This vignette approach to survey 
research is a common method used to approximate decision making within a heterogeneous 
respondent population (Alexander and Becker 1978; Mutz 2011). 

We ask respondents to recommend the major for Steve, rather than report their own 
preferences about majors, in order to allow us to isolate how earnings information affects the 
choice of major. In particular, there is a high degree of heterogeneity across respondents in tastes, 
academic ability, and private information about their own labor market prospects. Although 
randomization of the treatment guarantees that, on average, these traits will be balanced across 
the treatment and control groups, they introduce considerable variance into the estimates of 
treatment effects. Therefore, we ask respondents to instead focus on a fictional character, for 
whom we can control these characteristics. Although the answers apply to a fictional character, 
we still are able to assess how labor market information affects the evaluation of different 
majors. 13 Question wording was decided only after conducting pre-tests with Rutgers University 
students. 

To recruit a sample of respondents, we used Amazon’s Mechanical Turk (MTurk). MTurk 
is a platform for hiring people to work on computer-based tasks and is increasingly used to field 
social science experiments. 14 Our study is limited to U.S. adults and restricted to respondents 
with MTurk approval rates above 98%, which is an indicator of a respondent’s reputation for 

13 

Wiswall and Zafar (2013) employ a within-subject design to control for individual preferences. Our survey 

budget made this more time -intensive design infeasible. 

14 

Berinsky, Huber, and Lenz (2012) find that MTurk respondents generally replicate experimental results 
obtained from other convenience samples, such as college undergraduates. 
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quality work. Before obtaining consent, we cautioned respondents that we would not compensate 
them for survey responses that were completed too quickly to be reliable (e.g., surveys completed 
in less than 30 seconds). We also stated that any respondent would only be allowed to take the 
survey once (each respondent leaves a unique IP address). After giving consent, a link directed 
respondents to the online survey, which is hosted by Qualtrics. Respondents received a payment 
of 75 cents for completion of the survey. A total of 601 respondents completed the survey, with 
an average completion time of 6 minutes and 10 seconds. We remove 12 respondents from the 
final dataset who either finished the survey in too short a time to have fully read the survey 
instrument or were registered by Qualtrics to have taken the survey more than once. 

The sample characteristics deviate from national population characteristics, but reflect 
deviations commonly found in both adult and student convenience samples. 15 The median age of 
respondents is 28, with first and third quartiles equal to 24 and 34, respectively. Nearly 47% of 
respondents hold at least a college degree, and only 35% are female. 16 

One concern about this design is that by surveying adults we are missing how students 
evaluate labor market infonnation. We agree that students are a crucial consumer of labor market 
information, and ongoing projects are assessing the effects of labor market information on a 
student sample. But adults are also consumers of labor market outcomes data. In fact, outcomes 
data are frequently marketed to parents and college-bound students. Parents have an incentive to 
use outcomes data when recommending majors to their children, particularly since the parents 
often finance all or part of a college degree. Adults, more generally, use outcomes data when 


See Berinsky, Huber, and Lenz (2012) for characteristics of frequently used convenience samples. 

16 Median age of U.S population is 37 according to the 2012 U.S. Census. The 2012 American Community 
Survey reports that 28.5% of the population holds a college degree. 
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deciding whether to return to school and choosing a program of study. For these reasons, 
studying how a sample of U.S. adults evaluates this information is of substantive interest in itself 

One other concern involves the external validity of an experiment conducted on the 
MTurk non-probability sample. External validity requires us to consider to which population the 
experimental results generalize. Our convenience sample, as stated above, is more educated, 
younger, and male than the U.S. population as a whole. However, our primary goal in this paper 
is to estimate a causal relationship between earnings information and educational choice. 
Achieving that goal requires careful design of the experiment and randomization of the treatment 
among the respondents. In this analysis, we are not estimating the effect of earnings information 
on a given population, such as U.S. adults with children, or U.S. college -bound students. 
Understanding how these estimated effects generalize to different populations is a goal of future 
research. 

The survey instrument is based on a 2x2 factorial experimental design. The factorial 
design is defined as an experiment with two or more factors, in which respondents are randomly 
assigned to each possible experimental condition (Gerber and Green 2012). Its particular 
advantage is that it allows us to examine how the treatment effect (which labor market earnings 
information is displayed) changes depending on the values of the second factor (the achievement 
condition). 

The primary experimental factor involves the display of earnings information. The control 
condition for earnings information, which provides infonnation on average earnings alone, is 
shown in Table 1. The treatment condition for earnings information, which provides infonnation 
on average and the variation, or risk, in earnings, is shown in Table 2 and Figure 1. We present 
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data on two majors only: mathematics and economics. We choose these two majors because a 
quantitatively oriented student could reasonably choose either major, so our vignette contains a 
plausible choice for our fictional Steve. Second, the two majors have similar median earnings but 
much different variation in earnings. Economics with its wide earnings dispersion is a more risky 
major, whereas mathematics with its more narrow earnings distribution is a less risky major. 
Thus, the two majors provide an ideal test for the influence of showing the variation in earnings 
versus the median alone . 17 Respondents who are in the control group see the same table, but we 
show the median only . 18 We show the treatment condition for earnings data in both a table and a 
figure to make the data easier for respondents to interpret. In the data presented to respondents, 
we label median earnings as average earnings to simplify the understanding of this measure for 
respondents as a measure of central tendency, since average is more widely understood than 
median. Our calculations reveal very little difference between median and mean. 

The data presented in the tables and figure is calculated from New Jersey administrative 
data under the Workforce Data Quality Initiative. We link student enrollment records from New 
Jersey public universities and colleges to Unemployment Insurance wage records, which include 
nearly all graduates who work in New Jersey. We use New Jersey data, rather than national data, 
in order to provide more contextualized information about the possible earnings outcomes for 
Steve as is increasingly being done in states around the nation. In the survey instrument, we ask 
respondents to consider that these are earnings in Steve’s state, where he plans to live and work. 


17 

Of course, not all majors have comparisons this stark, which raises the concern that differences are not common 
and thus not of much importance. However, our concern is this paper is to show whether these differences can 
have an impact on student choice. For this reason, we choose two majors where variation in wages could lead to 
different student choices. 

18 

We indicate that wages are for graduates near age 30 so respondents don’t lower expectations based on the time 
after graduation needed to find a job or attendance in graduate school. 
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Thus, state-level earnings are a more accurate projection of labor market outcomes than national 
earnings. 


Table 1: Control condition for earnings information 

Major Category Average Earnings 


Mathematics 

$57,000 

Economics 

$55,000 


Table 2: Treatment condition for earnings infonnation 


Major Category 

25% Make Less Than 

Average Earnings 

25% Make More Than 

Mathematics 

$49,000 

$57,000 

$67,000 

Economics 

$34,000 

$55,000 

$73,000 


Fig- 1 : Visual display of data for the treatment condition for earnings information 



CD 

CD 

CD 

CD 

^r 


CD 

CD 

O 


CD 

CD 

CD 

O-J 


Annual Earnings 


16 


The secondary experimental factor is the achievement condition. We introduce this 
second factor by randomly assigning respondents to one of two prompts, which respondents read 
before proceeding to the earnings information. The prompts introduce information that can 
influence Steve’s propensity to realize higher earnings in the labor market. Specifically, we 
mention Steve’s academic ability, since many studies have shown that academic and cognitive 
ability influences labor market outcomes (e.g., Cawley, Heckman, and Vytlacil 2001). 19 We 
introduce this information in order to hold constant Steve’s private information about his 
prospects for success in the labor market as we vary the information we provide about earnings. 
We then are able to assess whether private information about labor market prospects influences 
the interpretation of earnings data. 

Two levels of the achievement factors — high and low — are shown below. In addition to the 
private infonnation, we also include other infonnation that holds constant factors related to major 
choice and earnings. In our experiment, we aim to highlight the influence of earnings information 
on major choice, and not school of graduation or preference for one major over the other. Thus, 
in each condition, we state that Steve is at a large public university, is choosing between two 
majors, and is likely to perform equally well in each major. This infonnation is intended to hold 
schooling and academic ability constant within each condition, so that the respondent does not 
recommend one major over another due to perceived opportunities available to graduates of elite 
schools or perceived chances of academic success. The last piece of information we provide is on 
the amount of student loan debt Steve will have. We hold this constant in all scenarios but 
include this information as a primer that earnings after college will be important for Steve. 


19 

While we also recgonize that other factors, such as social and family connections, are important, we do not 
examine these in this experiment because of limitations in scope of the study. 
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High Achievement: Steve is a sophomore at a large public university and is trying 
to decide on a major. He is choosing between mathematics and economics. He has 
always been a high achiever academically, and expects to perfonn equally well in each 
major. He will graduate with approximately $30,000 in student loans. 

Low Achievement: Steve is a sophomore at a large public university and is trying 
to decide on a major. He is choosing between mathematics and economics. He has 
always struggled somewhat academically, and expects to perform equally well in each 
major. He will graduate with approximately $30,000 in student loans. 

After we randomly assign individuals either to the high academic achievement 
condition or to the low academic achievement condition, we then randomly assign 
respondents into one of two conditions in the earnings information factor. In the first 
condition, which we call the low information condition, respondents view only the median 
earnings for both economics and mathematics. In the second condition, which we call the 
high information condition, respondents view the median earnings and the first and third 
quartiles of the earnings distribution for both economics and mathematics. 20 The first and 
third quartiles summarize the spread of the distribution, capturing the variation in earnings. 
Our experiment thus has four experimental conditions: high information and high academic 
achievement, high information and low academic achievement, low information and high 
academic achievement, and low information and low academic achievement. 

Each experimental condition is designed to assess our theoretical expectations described 
above. In particular, high information gives respondents accurate infonnation about the 

20 

In the survey instrument, we use the term “average” instead of “median” in order to avoid any possible 
confusion among respondents unfamiliar with the term “median.” 
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distribution, or risk, of earnings by major. Risk infonnation allows respondents to choose a major 
consistent with their risk preferences. In contrast, the low information condition forces 
respondents to recommend a major based on their own and often inaccurate estimate of the true 
risk associated with the major. The recommendation of major based on erroneous estimates of 
risk leads to respondents selecting majors that are inconsistent with their risk preferences. Across 
both high and low infonnation treatments, we randomize respondents by the high and low 
academic achievement conditions. By randomly varying levels of academic achievement, we 
assess whether high (low) academic achievement induces more (less) acceptance of the risk 
associated with a major’s labor market outcomes. 

After viewing the labor market infonnation in the experimental conditions, respondents 
report how much Steve is likely to earn in a given major, how likely Steve is to earn above 
$70,000 per year, how likely Steve is to earn below $38,000 per year, and whether they 
recommend that Steve pursue a given major. The first question assesses expected earnings in the 
major, while the second and third questions assess the risk associated with the major, in the fonn 
of high or low earnings, and the final question involves the educational choice of the 
respondent. 21 The full survey instrument is available in the appendix. 

We conduct randomization checks and find evidence that the randomization of 
information and ability treatments were successful. Across experimental conditions, we find no 
significant difference in the observable variables of gender, age, or education. In the treatment 
group, we find a slightly greater percentage of female respondents, but the difference is not 
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We choose the values $70,000 and $38,000 to represent typical high and low earnings outcomes for college 
graduates, based on the first and third quantiles of all graduates in our New Jersey administrative data. 
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statistically significant. When we include gender in the regressions presented below, we find 
substantively and statistically similar estimates. 


In our analysis, we focus on estimating the sample average treatment effect (SATE), or the 
average effect conditional on our sample of respondents. 22 Since we randomly assign the 
treatment to respondents, a simple difference-in-means estimator is an unbiased estimate of the 
SATE. However, to simplify presentation of results and to maintain a unified estimation strategy 
throughout the paper, we use simple linear regression to estimate the SATE. More specifically, 
we use the linear probability model to estimate the treatment effect of labor market outcomes 
information on major recommendation. Since the dependent variable is dichotomous, an 
alternative strategy is to use logit or probit regression. We instead use the linear probability 
model since it allows for easier and more direct interpretation of the estimates, while still being a 
consistent estimator of the SATE (Angrist and Pischke 2009). Moreover, simple linear regression 
of the outcome on an indicator, or dummy, variable for treatment assignment is equal to the 
estimate of the difference-in-means estimator (Wooldridge 2002). 

We also use the 2x2 factorial design to estimate the SATE given high or low levels of 
academic achievement. To estimate this SATE, we again use linear regression, including a 
dummy variable for high academic achievement and the interaction of high academic 
achievement and infonnation treatment status. This regression exactly reproduces difference-in- 
means estimates from a 2x2 table of means by information treatment status and academic 
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We focus on the sample, rather than the population average treatment effect, since our sample is a convenience 
sample and cannot be considered a representative sample from the population of U.S. adults. See Imbens (2004) for 
a discussion of the distinctions between the two estimands. 
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achievement level. Significance tests for estimates are based on large-sample approximations of 
the normal distribution and heteroskedasticity-robust standard errors. 23 

3 Results 

3.1 Recommendation of Major 

We begin by showing a visual summary of the responses. Figure 2 displays the proportion 
of respondents who recommended mathematics and economics, by achievement condition. The 
figure reveals a stark contrast between treatment and control conditions in both achievement 
levels. The respondents who view the median earnings only are much more likely to recommend 
mathematics. In the left panel, only 15% of respondents who see the median earnings recommend 
economics, the riskier major. The proportion is slightly higher in the right panel, where only 20% 
of respondents recommend economics, the riskier major. The respondents who view both the 
median earnings and the variation in earnings exhibit a much different pattern of major 
recommendation. In the left panel, nearly 40% of respondents recommend economics, which is a 
25 percentage point difference from the control condition. What is surprising is that the right 
panel shows a similar pattern, meaning that even for the low achievement condition, in which 
Steve may be more likely to earn below the median, respondents are selecting economics at a 
higher rate than the control group. We return to discuss this finding at the end of this section. 


23 

Researchers debate the use of large-sample approximations versus randomization procedures for statistical 
inference with experimental data (see Freedman (2008)]; Green (2009)]). Samii and Aronow (2012) show that the 
ordinary least squares variance estimate is equal to Neyman’s conservative randomization-based variance estimate 
and the constant effects randomization variance estimate when the size of treatment and control groups is the 
same. When sample sizes are different, and treatment effects are assumed constant, heteroskedasticity-robust 
standard errors are equal to Neyman’s randomization-based estimate. 
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Fig. 2: Proportion of responses in each experimental condition 


High Achievement Condition 


Low Achievement Condition 



ei Economics (high risk) 
o Mathematics (low risk) 




^ Economics (high risk) 
□ Mathematics (low risk) 



We now report estimates of the SATE. Table 3 shows the estimates from two regressions. The 

Treatment Only column reports the estimated SATE of -0.192, which is simply the coefficient on 

the Treatment indicator. In other words, the treatment effect of showing variation in earnings 

causes a 19 percentage point decrease in the probability of choosing mathematics, the less risky 

major. The regression estimates in the column Treatment by Achievement Levels analyze whether 

the treatment effect differs significantly between the high and low achievement. Since the 

Achievement variable is coded 1 for the high-achievement condition, the coefficient on 

Treatment indicates that for the low achievement condition, respondents were 14.7 percentage 
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points less likely to choose math than the control group. The coefficient on the interaction 
between Treatment and Achievement is the estimated difference in treatment effect between the 


high and low achievement groups. The estimate shows that the treatment group that saw the high 
achievement condition were less likely to select math (9.5 percentage points) than the treatment 
group presented with the low achievement condition, 


While the estimate for the interaction effect is in the right direction, it is not statistically 
significant at the 0.05 level. We also use an F-test to test the null hypothesis that the coefficient 
on Treatment x Achievement is equal to 0. The result from the F-test, which also fails to reject 
the null hypothesis, is reported in the lower section of Table 3. A summary of estimated treatment 
effects is reported in Table 4. 


Table 3: OLS estimates of linear probability models 



Treatment 

Treatment by Achievement 


Only 

Levels 

Intercept 

*** 

0.823 

*** 

0.797 


(0.022) 

(0.033) 

Treatment 

-0.192 

** 

-0.147 


(0.036) 

(0.050) 

Achievement 


0.044 

(0.043) 

Treatment x Achievement 


-0.095 

(0.073) 

F-statistic for 1/^: no interaction 


1.74 

/7-value of F-statistic 


0.187 

Observations 

589 

589 


**>0.001, *>0.01, 


>0.05 


Note: Dependent variable in each regression is equal to one if respondent recommended 
mathematics and zero otherwise. 


Table 4: Summary of average treatment effects 

Treatment Effect Estimate 
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SATE (High Achievement) 

-25% 

SATE (Eow Achievement) 

-15% 

SATE 

-19% 


Note: The SATE is statistically significant at the 0.01 level. The F-Test reported in Table 3 fails 
to reject the null hypothesis that the average treatment effect does not differ between the high and 
low achievement groups (F = 1.74,/; = 0.19). 


3.2 Information and Earnings Expectations 

A key component of the theories of student choice outlined in Section 1 .3 is that 
individuals base their major choice on their expectations of future earnings. Using those theories 
to develop empirical expectations, we suggest that the treatment induces a higher probability of 
choosing economics through its effect on higher earnings expectations. In other words, earnings 
expectations are the mechanism through which the treatment affects the outcome of major 
recommendation. An observable implication of this theory is that the treatment has a causal effect 
on earnings expectations. 24 While we do not claim to estimate the full causal chain — from 
treatment through earnings expectations to major recommendation — showing an effect of 
treatment on earnings is highly suggestive evidence in support of the theoretical expectations. 
Moreover, a finding of no treatment effect on earnings expectations would cast doubt on the 
theoretical expectations with regard to earnings uncertainty, and on the experimental 
manipulation itself. 

We use the three regressions to estimate the SATE of variation in earnings on earnings 
expectations. The dependent variable in these regressions is a relative measure of earnings 
expectations. For example, in column two in Figure 5, we take the percent chance of making over 

24 

In this analysis, we do not perform statistical mediation analysis. The methodological challenges faced in 
causal mediation analysis are discussed in Gelman and Hill (2006), Gerber and Green (2012), and tmai, Keale, 
Tingley, and Yanomoto (2011). 
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$70,000 in economics minus the percent chance of earning over $70,000 in mathematics. The 
relative measure captures the idea that being more likely to achieve higher earnings in economics 
causes respondents to favor that major, relative to mathematics. Positive values of the variable 
represent respondents who expect to earn more in economics than in mathematics, while negative 
values represent respondents who expect to earn more in mathematics than in economics. 25 In 
column four in Table 5, we calculate the difference in expected average earnings between 
economics and mathematics. Together, the three measures capture the treatment’s effect on 
expected average earnings (column 4) and risk (columns 2 and 3). In other words, each measure 
corresponds to a different aspect of the theory of student choice: how likely the student is to 
obtain a high-earnings outcome, how likely Steve is to obtain a low-earnings outcome, and how 
much money the respondent believes Steve is most likely to earn. 

We report the results from all three regressions in Table 5. We find that the treatment 
effect has a positive and statistically significant effect on the relative percent chance of high or 
low earnings. Showing respondents the variation in earnings causes nearly a six percentage point 
(5.91) increase in the difference between economics and mathematics among the low 
achievement group. The low achievement group that received the treatment was also more likely 
(7.00 percentage points) to report earnings for Steve below $38,000, compared to the low 
achievement group that viewed average earnings alone. 

For the high achievement group, respondents were somewhat more likely (1.14 
percentage points) to report that Steve will earn above $70,000, though this difference is not 

25 

We also conducted these regressions using the probability of high or low earnings for each major alone; that is, 
we did not create a relative measure of earnings expectations. The treatment effect is large and statistically 
significant. However, we do not report these results since they cannot account for the relative preference of one 
major over another. 
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statistically significant. Respondents in the high achievement group were also somewhat less 
likely (-1.13 percentage points) to report that Steve will earn under $38,000, though again this 
difference is not statistically significant. The null finding with regard to academic achievement is 
consistent with our analysis of major choice, where we also find estimates in a direction 
consistent with the empirical expectation but not statistically significant different across 
achievement levels in the choice of economics and mathematics. 


Respondents in the treatment condition, in other words, expected higher earnings in 
economics relative to mathematics. Similarly, respondents in the treatment group also expressed 
a greater chance of earning less in economics: the difference between treatment and control 
groups is seven percentage points. 


Table 5: Ordinary least squares estimates of treatment and achievement level 



Relative % Chance 

Relative % Chance 

Expected 


Earning over $70,000 

Earning under $38,000 

Earnings 

(average) 

Intercept 

-1.46 

-0.18 

-2355 


(0.94) 

(0.87) 

(556) 

Treatment 


7 00*** 

-879 


(1.19) 

(1.39) 

(-738) 

Achievement 

0.42 

0.63 

720 


(1.32) 

(1.15) 

(875) 

Treatment x Achievement 

1.14 

-1.13 

109 


(1.85) 

(1.79) 

(1214) 

Observations 

589 

589 

589 

F statistic for nested models 

0.39 

0.38 

0.008 

p-value of F statistic 

0.53 

0.54 

0.93 


**Vo. 001 , * Vo-01, Vo.05 

Note: Dependent variable in second column is the percent chance earning over $70,000 in economics 
minus the percent chance earning over $70,000 in mathematics. Dependent variable in third column is the 
percent chance earning under $38,000 in economics minus the percent chance earning under $38,000 in 
mathematics. Larger values indicate respondent estimates of greater variation in earnings for economics, 
relative to mathematics. The dependent variable in the fourth column is the difference in expected average 
earnings between economics and mathematics. 
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Treatment group respondents’ estimates of a greater chance of high and low earnings 
reflect the higher variation of earnings in economics; respondents are reporting expectations that 
are consistent with the data. The control group responses provide a stark contrast. In the control 
group, respondents report almost no difference between the two majors. In column two, the 
control group average response is only a difference of -1.46 percentage points; in column 3, the 
control group average response is only -0.18. These small differences are evidence that 
respondents have poor estimates of the true variation in earnings in these majors. In particular, 
these small differences are consistent with respondents estimating similar variation in earnings 
across the two majors, while the real variation in earnings differs considerably. 

The results, taken together, support the theoretical expectation that the treatment 
influences major choice through its effect on earnings expectations. Respondents use the 
information about variation to estimate probabilities over high and low future earnings outcomes. 
The treatment effect of information on major recommendation appears to be driven by the 
treatment causing respondents to estimate higher probabilities of earning high and low earnings, 
and not driven by changes in expected earnings. We find no evidence that the treatment effect is 
driven by changes in expected earnings: in column three, the coefficient on the treatment is small 
(-$878) and statistically indistinguishable from zero. 

4 Discussion 

In Section 1, we use economic theories of student educational choice to form three 
empirical expectations. Our first empirical expectation concerns the recommendation of major. 
Based on the theoretical and empirical work discussed in Section 1, we expect infonnation about 
earnings variation to change, on average, the recommendation of major through its effect on 
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earnings expectations. While we cannot identify this complete causal chain — from treatment to 
earnings expectations and major choice — with our experimental design (see Imai, Keele, 

Tingley, and Yamamoto 2011), we do report findings consistent with our expectation and the 
theories cited in Section 1 . 

Our second empirical expectation was that providing individuals with infonnation about 
the variation in earnings leads to different earnings expectations compared to individuals who see 
median earnings alone. We find strong evidence in support of this expectation. Respondents 
observed the high variation in earnings for economics and used that information to report both a 
higher percent chance of earning more than $70,000 and a higher percent chance of earning less 
than $38,000. In other words, the treatment induces both more optimistic and pessimistic 
expectations about earnings. 

We find no difference with regard to changes in average expected earnings. Respondents 
in both experimental groups are highly likely to choose at or near the median when asked to 
report how much Steve is likely to earn. We find no evidence that respondents use information 
about the variation in earnings to update the expected, or most likely, earnings amount. 

This response pattern is consistent with individuals using earnings data to update their 
own erroneous prior beliefs about population earnings (Wiswall and Zafar 2013). Respondents in 
the control condition did not see the distribution in earnings and thus used their own prior beliefs 
to guess about the probability of achieving earnings above $70,000 and below $38,000. These 
prior beliefs appear to have been based on underestimated estimates of the variation of earnings 
in economics. 
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Our third empirical expectation concerns the role of academic achievement as a 
mechanism to explain why the interpretation of earnings data varies across individuals in our 
experiment. We find limited evidence of an effect for infonnation about academic ability. In both 
the analysis of major choice and earnings expectations, the estimated differences in responses 
across academic achievement conditions were in the direction consistent with the empirical 
expectations but statistically insignificant at conventional levels. 

Given research on the relationships between academic ability, major choice, and labor 
market outcomes (Arcidiacono 2004), we are surprised to not find strong evidence that 
respondents use information about academic ability to estimate future labor market outcomes. 
This null finding can be a result of several factors. Respondents may not believe there is a strong 
linkage between academic performance and post-graduate earnings. Respondents may also be 
influenced by optimism bias, discounting negative outcomes in both high and low academic 
achievement conditions. Finally, our academic achievement treatment may not be strong enough 
to cause respondents to believe that Steve’s academic performance will affect his post-graduate 
earnings. Our treatment states that Steve “struggled somewhat” academically. Respondents 
possibly don’t consider “struggled somewhat” to influence post-graduate earnings as much as, for 
example, a treatment that stated Steve was a significant poor perfonner in school. 

The statistically significant treatment effects we report are average differences. Though 

the treatment effect of earnings infonnation on major choice is quite large, the majority of 

respondents continue to choose mathematics in the treatment condition. Why, despite the promise 

of higher earnings in economics, do a majority of respondents recommend the less-risky major 

with less potential for higher earnings? It is likely that other mechanisms involved in 

respondents’ interpretation of the earnings data explain this variation in responses. These other 
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mechanisms include class-based factors such as social networks and/or economic resources, as 
well as or psychological traits such as risk-aversion and decision making biases. Our results are 
consistent with both risk-aversion and decision making biases influencing the earnings 
expectations and major choice. Risk-averse respondents recommend mathematics given the 
disutility of the low-eamings outcome in economics, while respondents influenced by optimism 
bias discount the higher probability of a low-eamings outcome in economics; respondents who 
do not discount the higher probability of the low-eamings outcome choose mathematics. 

Our results are consistent with previous work analyzing risk and educational choice 
(Nielsen and Vissing-Jorgensen 2006; Wiswall and Zafar 2013). Overall, respondents in our 
sample prefer the major with less risk — mathematics. However, we show that a significant 
number of respondents prefer the high-risk major of economics when exposed to the variation 
treatment. This result shows that not all respondents prefer high-mean, low-risk majors. Rather, 
some respondents, possibly those with risk-seeking preferences or optimism bias, prefer the 
major with high risk. 

Finally, our data provide limited information on the role of class-based factors. Our 
preliminary analyses of respondents of different class backgrounds suggest that the results vary 
by social class, when class is measured by parents’ educational attainment. Future work should 
identify exactly how social class influences choice in the context of labor market information 
using more encompassing measures of social class. 

5 Conclusion 
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Incorporating a simple model of student choice under uncertainty reveals some puzzling 
implications about common methods used to display outcomes data. Specifically, showing 
students the median only assumes a model of student choice in which students make decisions 
based on the future expected earnings only, ignoring the risk, or variance, of future earnings. This 
model of student decision making is equivalent to a model of investor decision making in which 
investors optimize their portfolio earnings without regard to risk preferences or the riskiness of 
the assets. We believe that the results in this paper show that an explicit theory of student choice 
is missing from discussions about the provision of labor market outcomes data. This is surprising, 
given extensive work in the social sciences documenting how expected earnings influence 
educational choices (Arcidiacono 2004; Arcidiacono et al. 2012; Becker 1994; Willis and Rosen 
1979; Wiswall and Zafar 2013). One contribution of our experiment is a better understanding of 
the effects of omitting infonnation on risk from scorecards, which presumes the mean-only 
model of student choice. 

Based on the findings from this research, it is clear that infonnation on the variation in 
earnings is an influential piece of infonnation in the decision making process. Further research on 
this specific issue needs to be conducted with direct consumers of this information, including 
college students, their parents, and counselors who advise students and parents. In addition, given 
the potentially important role that infonnation on risk as represented by variation in earnings has 
on low-SES students, further study specifically focused on these students is especially important. 
This future research may most effectively be conducted by examining the use of these tools in 
practice, including more in-depth understanding of how consumers react to these data. 

Beyond the specific issue of whether to present earnings data as a median or a range, 
many other important decisions related to how these data are presented need attention. These 
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decisions include the timing of the data relative to graduation, whether to provide the 
employment rate after graduation, which sub-group of students to report on, whether to report the 
data by college or program level, and whether to report on all students, including both graduates 
and non-graduates (Hannon, Ridley, and Zinn 2014; Moore et al. 2013; Voight et al. 2014). 

Targeted research using experimental studies can shed light on these decisions around 
how to present data. Broader research on students’ decision making processes and how these data 
infonn their decisions (or not) is essential to help guide the development of useful and relevant 
data products. To achieve the goal of more informed decision making through better information 
rests entirely on ensuring the quality of that information. Further research can ensure that quality 
infonnation achieves this goal. 
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